Comparing grammar-based and robust approaches to speech understanding: a case study

Previous work has demonstrated the success of statistical language models when enough training data is available [1], but despite that, grammar-based systems are proving the preferred choice in successful commercial systems such as HeyAnita [2], BeVocal [3] and Tellme [4], largely due to the difficulty involved in obtaining a corpus of training data. Here we trained an SLM on data obtained using a grammar-based system and compared the performance of the two systems with regards to recognition. We also parsed the output of the SLM using a robust parser and compared the accuracy of the semantic output of the systems. The SLM/robust parser showed considerable improvement on unconstrained input, and similar precision/recall (per slot value) on utterances provided by trained users.